1. Field of the Invention
The present invention relates to a method and apparatus for smoothly merging data such as image data. The present invention also relates to a medium for providing a data merging program for the data merging method.
2. Description of the Related Art
Multi-resolution-based techniques are available as a method for splicing two or more images into a single image having a smooth splice. These techniques are described in a paper authored by Peter J. Burt et al., entitled xe2x80x9cA Multiresolution Spline With Application to Image Mosaics,xe2x80x9d ACM Trans. on Graphics, Vol. 2, No. 4, October 1983, pp. 217-236, and in a paper authored by Peter J. Burt, entitled xe2x80x9cFast Filter Transforms for Image Processingxe2x80x9d, Computer Graphics and Image Processing, 16, 1981, pp. 20-51. The method using the multi-resolution techniques described in these papers include following three steps.
In a first step, each image is divided by spatial frequency band. A set of divided images is called a Laplacian pyramid. In a second step, the images of different frequency components are blended for splicing according to weighted average. In a third step, the set of images spliced according to the frequency component is summed to form a merged image.
The Laplacian pyramid is a set of images that are divided according to spatial frequency band. A method of creating the Laplacian pyramid is briefly discussed.
An original image is low-pass filtered using a reduce operation to produce a set of images called a Gaussian pyramid. The reduce operation here refers to an operation in which the original image is low-pass filtered to reduce a sample density of the image.
When the original image is subjected to the reduce operation once, image data at a first layer of the Gaussian pyramid is created. Another reduce operation, if executed, creates image data at a second layer of the Gaussian pyramid. The reduce operation is iterated to create image data of further layers. The original data is called a zero-layer image data. The higher the hierarchical layer in the Gaussian pyramid, the lower the frequency component of the image becomes as a result of low pass filtering.
Using the Gaussian pyramid thus constructed and an expand operation, the Laplacian pyramid is created. The expand operation is an operation reverse to the reduce operation. The xe2x80x9cimage data at the i-th layer of the Laplacian pyramidxe2x80x9d is created by subtracting the xe2x80x9cimage data that is obtained by subjecting once the image data at (i+1)-th layer of the Gaussian pyramid to the expand operationxe2x80x9d from the xe2x80x9cimage data at an i-th layer of the Laplacian pyramidxe2x80x9d.
The image data at each layer in the Laplacian pyramid is the one, having a particular frequency band, picked up from the original image data. As the layer moves upward in the Laplacian pyramid, the image has a lower frequency band.
The Laplacian pyramid, the Gaussian pyramid, the reduce operation and the expand operation have been discussed. A detailed discussion about them is provided in the two above-cited papers.
The image merging using the multi-resolution techniques disclosed in the above-cited papers is now discussed.
The images are merged on the basis of the frequency component according to the cited merging method. The size of each merging area is different from frequency component to frequency component. In a low-frequency component, namely, a high layer in the Laplacian pyramid, two images are blended with respective weights in a wide merge area, while in a high-frequency component, namely, a low layer in the Laplacian pyramid, two images are blended with respective weights in a narrow merge area. The merge area in the low-frequency is set to be wide because of the following reason: if the images having a direct current component (the extreme case of low-frequency components) are spliced in a narrow area, a discontinuity stands out, and the images are thus allowed to mildly vary in a wide area for splicing. The merge area in the high-frequency component is set to be narrow. The high-frequency components at an edge or other features, if blended in a wide area, are blurred and look like double exposures.
In accordance with the merging method using the multi-resolution techniques, in the low-frequency component, namely, the high layers in the Laplacian pyramid, two images are blended with respective weights in a wide merge area. The lower the frequency component, the wider the merge area. At the lowest frequency band, namely, in the topmost layer in the Laplacian pyramid, the size of the merge area extends to the entire original image.
If images formed of elements having extremely varying aspect ratios, such as images of the hair, are merged using the above-referenced merging method, a natural looking merging is difficult.
Now, one single hair stands out among a plurality of hairs. The outstanding one single hair means that the edge of the one single hair looks distinct because of the lighting conditions. Such a hair contains a great deal of high-frequency components, and data about it is placed in a low layer in the Laplacian pyramid. The hair is thus blended in a narrow area in the center of the merge area. Specifically, the above merging method results in an unnatural-looking image in which the outstanding hair extending from one side disappears abruptly in the center.
Now, an original image shown in FIG. 1A and an original image shown in FIG. 1B are merged at the center of the screen.
FIG. 1A and FIG. 1B are images of elements such as the hair, featuring extremely varying aspect ratios. Some elements (hairs) are distinct while other elements (hairs) are less distinct.
Referring to FIG. 1A, black stripes (a1, a2, a3 and a4) are distinct elements (hairs). The blank portion indicates less distinct elements, which are not shown in FIG. 1A because they are not important in the context of the present invention. Similarly, black stripes (b1, b2, b3, and b4) shown in FIG. 1B are distinct elements (hairs). The blank portion indicates less distinct elements, which are not shown in FIG. 1B because they are not important in the context of the present invention.
FIG. 1C shows a merged image C, into which the original image shown in FIG. 1A and the original image shown in FIG. 1B were merged in accordance with the conventional data merging method. Since the distinct elements (hairs) contain a great deal of high-frequency components in the merged image, the elements abruptly increase in number near the center (at a vertically aligned dotted line cc, which does not appear in the real image).
Elements a1, a2, and a4 shown in FIG. 1A become ca1, ca2, and ca4, respectively, as shown in FIG. 1C, and ca1, ca2, and ca4 appear only on the left half of the screen. Since a3 does not appear on the left half as shown in FIG. 1A, it does not appear in FIG. 1C, either. Elements b1, b2, b3, and b4 shown in FIG. 1B become cb1, cb2, cb3, and cb4 as shown in FIG. 1C, and cb1, cb2, cb3, and cb4 appear only on the right half.
Accordingly, it is an object of the present invention to provide a method and apparatus for merging a plurality of data in a natural looking manner.
It is another object of the present invention to provide a medium for providing a program for the data merging method.
According to one aspect of the present invention, a method for merging a plurality of data includes the extracting step of extracting at least one of the high-frequency component of first data and the high-frequency component of second data, and the synthesizing step of determining a composite value of the first data and the second data in accordance with at least one of the amount of the high-frequency component of the first data and the amount of the high-frequency component of the second data.
According to another aspect of the present invention, an apparatus for merging a plurality of data, includes an extracting unit for extracting at least one of the high-frequency component of first data and the high-frequency component of second data, and a synthesizing unit for determining a composite value of the first data and the second data in accordance with at least one of the amount of the high-frequency component of the first data and the amount of the high-frequency component of the second data.
According to yet another aspect of the present invention, a medium for provides a program for merging a plurality of data. The program includes the extracting step of extracting at least one of the high-frequency component of first data and the high-frequency component of second data, and the synthesizing step of determining a composite value of the first data and the second data in accordance with at least one of the amount of the high-frequency component of the first data and the amount of the high-frequency component of the second data.